Knowledge-Constrained Answer Generation for Open-Ended Video Question Answering
نویسندگان
چکیده
Open-ended Video question answering (open-ended VideoQA) aims to understand video content and semantics generate the correct answers. Most of best performing models define problem as a discriminative task multi-label classification. In real-world scenarios, however, it is difficult candidate set that includes all possible this paper, we propose Knowledge-constrained Generative VideoQA Algorithm (KcGA) with an encoder-decoder pipeline, which enables out-of-domain answer generation through adaptive external knowledge module multi-stream information control mechanism. We use ClipBERT extract video-question features, framewise object-level from commonsense base compute contextual-aware episode memory units via attention based GRU form exploit mechanism fuse features such semantic complementation alignment are well achieved. evaluate our model on two open-ended benchmark datasets demonstrate can effectively robustly high-quality answers without restrictions training data.
منابع مشابه
Open-Ended Visual Question-Answering
This thesis studies methods to solve Visual Question-Answering (VQA) tasks with a Deep Learning framework. As a preliminary step, we explore Long Short-Term Memory (LSTM) networks used in Natural Language Processing (NLP) to tackle Question-Answering (text based). We then modify the previous model to accept an image as an input in addition to the question. For this purpose, we explore the VGG-1...
متن کاملProposing Plausible Answers for Open-ended Visual Question Answering
Answering open-ended questions is an essential capability for any intelligent agent. One of the most interesting recent open-ended question answering challenges is Visual Question Answering (VQA) which attempts to evaluate a system’s visual understanding through its answers to natural language questions about images. There exist many approaches to VQA, the majority of which do not exhibit deepe...
متن کاملAnswer Formulation for Question-Answering
In this paper, we describe our experimentations in answer formulation for question-answering (QA) systems. In the context of QA, answer formulation can serve two purposes: improving answer extraction or improving human-computer interaction (HCI). Each purpose has different precision/recall requirements. We present our experiments for both purposes and argue that formulations of better linguisti...
متن کاملQuestion Generation for Question Answering
This paper presents how to generate questions from given passages using neural networks, where large scale QA pairs are automatically crawled and processed from Community-QA website, and used as training data. The contribution of the paper is 2-fold: First, two types of question generation approaches are proposed, one is a retrieval-based method using convolution neural network (CNN), the other...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Proceedings of the ... AAAI Conference on Artificial Intelligence
سال: 2023
ISSN: ['2159-5399', '2374-3468']
DOI: https://doi.org/10.1609/aaai.v37i7.25983